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Abstract - In order to locate a robot band accurately, a vision 
sensor play a key role by measuring relative position. In our sys- 
tem, we assume that two 5-DOF manipulator fit a bolt and nut in 
cooperation. However, a vision sensor on a band can not measure 
the position of an object which ii grasped by the hand. In this 
method, a vision sensor is installed on the hand of each robot, and 
then measure a position of the band and the object each other. 
Finally, each vision sensor compensate hand position each other. 

1. Introduction 

The conventional industrial robot is suitable to repealed work based 
on the teaching play back. However, in the case of the off-line work 
based on CAD or so called intelligent system, difficult problems occur 
such that there are some errors between planned trajectory based on 
an internal model and actual trajectory. In such work, it is necessary to 
develop some sensor for recognising an error. Several calibration 
method [ 1 J -[7] using vision , ultrasonic and so on have been developed 
up to date. Additionally, a method which directory correct the hand 
position using vision sensor with no calibration are reported. 

Thus, In order to locate a robot hand accurately, a vision sen- 
sor play a key role by measuring relative position between a hand and 
an object. We have developed a compact vision sensor[8] which is 
easily installed on a hand of a robot. However, when several robots 
work in cooperation, a vision sensor on a hand can not measure the 
position of an object which is grasped by the hand because of dead 
angle and ochision 
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Fig.l Experimental set up of the system 
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In this paper, we propose a novel method for measuring a rela- 
tive position between a hand and an object using a vision sensor 
which is installed on u hand of the other robot. Using the vision sen- 
sor, we can improve depth measurement accuracy. 

2. Definition of a task and model 

In this experiment, we assume that two 5-DOF manipulators fit a bolt 
and nut as shown in fig. 3. A vision sensor is installed on a hand of 
each robot, and then measure a position of the hand and the object 
each other. Fig. 2 shows the coordinate systems defined on each joint 

ff 

of the manipulators. A position vector r on the object which is 
grasped by other robot is transformed into the sensor coordinates us- 
ing next equation. 

V = % '% % % % % T V W T B % XX X V r 

Manipulator 1 Manipulator 2 

Where l Tj is a homogeneous transformation matrix between coordi- 
nates i and j* Usually, the Denaveit-Hartenberg method is used for 
describe the model, bul, in our method, we use the homogenous trans- 
formation matrix. 

Actually, there are several errors, for example parameter 
identification error, deflection of manipulator arm, backlash and so 
on, so it is difficult to correct the positioning error by the conventional 
method. 

3. Principle of compensation. 

The vision sensor which is used in this method is composed of mul- 
tiple beam projector and CCD camera. This sensor can get a gray scale 
image and discrete range data at same position. 

3.1 Model description 

In the case of object recognition in actual environment, it is conve- 
nient to use model data of an object. In this method, we use the list 
structure for describing an object as shown in fig. 3. This model de- 
scribes a reliability factor R. in addition to face, vertex and point, 

because reliability of edge extraction is influenced by edge condition 
such as a rounded edge. 

3.2 Process of gray scale image 

In this paragraph, process of gray scale image is described. First, gra- 
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Scnsor2 Coordinates 



Robot2-base Coordinates 



Model Coordinates 



Sensorl Coordinates 




Robot 1 -base Coordinates 



Fig.2 Coordinate system define on each joint. 
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Fig.3 List Structure of Object Data 
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dienl vector of two images which duTer in li jhting position are ex- 
tracted. Next, model data are projected into image plane of the CCD 
camera using nominal parameter of the robot configuration. As shown 
in fig-4, a processed area is restricted under a condition that it is near 
area from an edge of the model and has same gradient values of the 
model. And then, extracted edges are fit into line and intersecting 
points are calculated using these lines. 

3.3 Compensation of hand position 

Position U of the model on the image plane of the sensor is calculated 
by next equation. 



u= m T*r 



where 
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and f is focal length of the 



sensor. 

However, because of kinematic errors of the robot, we add a 
mainx for compensating these error. 

u^TJiqfr (3) 

where g T = [ x y z <x P y ] r is translation and rotation along X Y 

and Z axes. If there arc some errors All between Hand measured 
position, and the All is assumed that All is caused by the T{q) , 
then All is calculated by next equation 

Au = jAq (4) 

where J is jacobi matrix defined by next equation. 



j _ du 



(5) 



(2) 
S4S 



Using an inverse matrix of this J and measured All, hand position 
is compensated. 
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Extracted edge 




Hill 

Fig4 Mode] Edge 



As shown in fig. 5, image data are obtained b\ the vision 
sensor in the task assumed in our experiment. Using this tmape. error 
An is calculated. 

First, edges are extracted using a model data as mentioned 111 

3.2. Comparing the calculated point with the model, Al/ is calcu- 
lated. Reliability of extracted edges are depend on edge length. So we 

define a reliability factor A- according to edge length 

Additionally, in the case of rounded edge, reliability is low no 

matter what edge length is enough. Therfore; wc define a lac tor /? tn 

the model data. 

Using these factor, factor >v. calculated using next equation 

yy t = A i B l i- 1, •;•,/! (<"»> 

And then, weight vv. is normalized using next equation 

^=»yl> i = ],...,„ ( 7, 

Using this weight w , equation (4) is extended into as follows 









w 2 Ait, 













) 



Aq 



(8) 



Finally, error parameters are calculated using next equation 




» '; Projected lasei spots 
Fig. 5 Model Fitting 




Senior Coordinates 



Fig .6 Fitting on Image Plane 




3D space 



FC plane 



Sensor Coordinates 



*9 = JS 



(9) 



Fig. 7 Fitting to Extracted Plane. 

where, J * is pseudo inverse matrix according to the 
weight w r 

The robot hand position can be compensated using this A^ - 
3.4 Use of discrete range data 

To improve an accuracy of depth measurement, we use the discrete 
range data obtained by the sensor. Fig. 5 is illustration of input image 
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when multiple laser beams arc projected. As shown in fig. 6, when an 
edge line is extracted, position error between model and an object is 
corrected into the plane which includes origin point of the sensor co- 
ordinates and edge line, so we can not expect high measurement accu- 
racy of depth. In our method, we improve the depth accuracy usine 
spot images shown in fig. 5. Three Dimensional range information is 
obtained from the spot image. First, FG plane parameters are esti- 
mated by the Least Mean Square method using spots which arc on a 
surface of the object And then. Candidates obtained by the method 
mentioned above are fit into the FG plane as shown in fig. 7. 

4. Experimental results 
An object used in this evaluation is a block with a size of 30 mm. 
Measurements are done at total of hundred positions which are in an 
area of 25 mm\ Errors of these measurements are indicated in 
fig. 7,8,9 and 10. Fig. 7 and 8 show the error obtained by the gray 



scale image only and fig. 9 and 10 is the image and discrete ranee 
data. These results say that distribution of depth (measurement error 
of Z-Direction) is improved from 10mm to about 1mm. Fig. 12 shows 
the model image which is superimposed on the hand and nut image. 
Before the compensation, these do not fit each other, hut after the 
compensation, they fit mutually. Thus performance of the proposed 
method is verified. 

4. Conclusion. 

In this paper, we have shown the method of compensation of robot 
hand position using vision sensor installed on the hand. Additionally, 
we show the performance of the method by the actual experiment 
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Fig .8 X-Y distribution of measurement error difference 
(Gray scale image) 
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Fig 1 0 X-Y distribution of measurement error difference 
(Gray scale image and discrete range data) 
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Fig .9 Z-Y distribution of measurement error difference 
(Gray scale image) 
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Fig . 1 1 Z-Y distribution of measurement error difference 
(Gray scale image and discrete range data) 
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